chore(drift-sync): needs-human model-family decision (2026-08-04) - #350

…amilies (wrong modality) (#343)

<!-- drift-changeset: eaa8db65f5647493 -->
<!-- drift-proposal-note:
drift-proposals/openai-gpt-live-transcribe-new-family.md -->
<!-- drift-proposal-note:
drift-proposals/openai-gpt-transcribe-new-family.md -->

> **⚠️ Do not remove the HTML comment markers above.** `fix-drift.yml`
dedups by grepping open PR
> bodies for `drift-changeset: <key>` and `drift-proposal-note: <path>`.
Removing them makes the next
> scheduled run treat this changeset as unproposed and open a duplicate
needs-human PR — which is
> exactly what happened when this body was rewritten (see #350).

Classifies the two OpenAI model families that shipped 2026-07-28 —
`gpt-transcribe` and `gpt-live-transcribe` — and pins the registry
surfaces that classification touches.

## The decision

Both families are **transcription-only**. `gpt-transcribe` serves
`/v1/audio/transcriptions` and realtime transcription sessions;
`gpt-live-transcribe` is realtime-only. **Neither answers on
`/v1/chat/completions`.**

- **`excludeFamilies.openai`** (`src/__tests__/drift/model-registry.ts`)
— the text-listing canary. Excluded, because they are the wrong
modality. This mirrors the existing entries for `gpt-4o-transcribe`,
`whisper-1` and `gpt-realtime*`, and follows established prior art for
keep-but-not-live families (`gemini-pro`, `gpt-3.5`).
- **`knownVoiceModelFamilies`** (`src/__tests__/drift/voice-models.ts`)
— the voice/realtime canary, a deliberately disjoint registry. Added,
because that is the remediation the collector itself prescribes:
*"Unknown realtime model detected — add to `knownVoiceModelFamilies`"*.

These are two different registries with two different jobs. Excluding
from the text canary is not the same as silencing detection — the voice
canary is where these families are tracked, and membership there is the
tool's own sanctioned resolution of the finding it raised.

The `Decision:` lines in the two `drift-proposals/` notes are
deliberately **not** set to `include`. The drift-sync auto-apply path
writes exclusively into `includeFamilies`, so following the note's own
boilerplate instruction would misclassify both families as chat-mocked.
The decision is recorded as prose; wiring an `exclude` decision value is
tracked separately.

## Live evidence

`pnpm test:drift` against the real provider APIs on this branch: **136
passed, 0 failed.**

Both canaries that are **red on `main`** are **green on this branch**:
- `✓ live /models contains no unclassified family`
- `✓ canary: GA realtime models available`

So this branch alone resolves the drift. `drift-live-pr` passes in CI.
Note that the `drift` job itself is gated `if: github.event_name !=
'pull_request'` and therefore **skips on PRs** — a live `Drift Tests`
dry-run on the final sha is the pre-merge gate, confirming the two
families no longer flag, the realtime canary is in the expected state,
and nothing else started flagging.

## What is in this PR, stated plainly

8 commits, **9 paths**. It is not a small diff: roughly **200 lines are
pin machinery** (`logic-pin.test.ts` is `+193/-1`), with the
classification itself being a handful of lines. Included deliberately
rather than deferred, because two of these pins were **mandatory
findings from review**:

- `excludeFamilies.openai` `DATA_FROZEN` re-pin — required by the
membership change.
- Pins for `knownVoiceModelFamilies`, `gaRealtimeModels` and
`isVoiceModelId`, plus the two functions that consume the seed sets.
Before this, **adding a family to the voice registry tripped no guard at
all**, and neutering `detectVoiceModelDrift` to `hasGA = true` — which
permanently disables the "provider removed a GA realtime family" alarm —
passed the entire 4,788-test suite with nothing watching it.
- An anchored assertion so the live text-lane canary cannot be silenced
by neutering its wrapper.

All pins are `^`-anchored and use behavioural anchors alongside
checksums: a checksum reports "something changed", a behavioural anchor
reports "this no longer detects an unclassified family". Every one was
proven by mutation — the guarded value was broken, the guard observed
failing, the mutation reverted, green confirmed. A pin nobody has
watched fail is not a guard.

Also: un-mangles the `src/__tests__/` paths cited in the two notes
(Prettier had eaten the underscores, so the committed audit trail cited
files that do not exist), adds `drift-proposals/` to `.prettierignore`
so that cannot recur, and drops two `gemini-1.5` notes whose decision
was already applied in `a3dc250`.

**This PR touches zero files under `scripts/`.** That is load-bearing:
it means the change cannot alter delta keys, so no base/head skew and no
transitional gate failure is possible.

## Split out of this PR

This branch previously carried 19 commits. The collector, delta-keying
and test-tooling work has been moved out, because three review rounds
each found real mandatory defects and almost all of them were in guards
the review loop itself had added — a scope signal, not a quality signal.

- **#349** (draft) — drift guard/tooling hardening: collector
remediation strings, delta-key provenance, the remediation-strings
guard, and the transition-window documentation. Expected red on the two
canaries inherited from `main` until this PR lands.
- **A third PR, still to open** — a confirmed **fail-silent** defect in
`scripts/drift-delta.ts`: the delta key omits `scenario`, so
same-provider entries collapse and genuinely new drift is downgraded
from `block` to `advisory`. Demonstrated on a real run: `block: []` with
a new-in-head **critical** sitting in `advisory`, gate passing.
Pre-existing, independent of this change, and more serious than anything
in either of the other two PRs.

## Recovery record

**Pre-strip head: `146cb9d2b81017097480b0616d36b576411aa11c`** (19
commits from the merge-base). The guard commits were pushed to
`drift/guard-hardening` **before** this branch was rewritten, and the
force-push was lease-pinned to that sha. Nothing was destroyed.

This PR is deliberately **not closed**: drift-sync's dedup is
open-PR-only, so closing it would re-arm the daily needs-human PR this
thread exists to resolve.


---

## Round-4 additions — the classification is now actually tested

Review found that the two families this PR exists to classify had **no
behavioural coverage at all**: remove them and only the membership
checksum reddened. A checksum reports "something changed"; it does not
report "these two are excluded and `gpt-live` is not." Four tests now
assert what the classification *means*, and they live in the **default**
suite — so they gate every PR rather than only running when someone
triggers the live drift job:

- dropping `gpt-transcribe` reds its own test; dropping
`gpt-live-transcribe` reds its own test
- a substring-style exclude match reds the boundary on `gpt-live`
- a prefix-style match reds it in reverse on `gpt-live-transcribe-mini`

That boundary matters concretely: `gpt-live` and `gpt-live-transcribe`
are different families, and a substring bug of exactly that shape was
found elsewhere in this repo during review (a provider-label fallback
resolved `"Gemini Live Transcription session"` to `Transcription`).

**The live-canary anti-silence work has been REMOVED from this PR and
moved to #349.** An earlier revision of this branch carried an anchor
intended to stop the live text canary being silenced. It was defeated at
four successive surfaces — the detector function, the gate deciding
whether it runs, the call site feeding it, and finally the fetcher the
leg calls — each fix closing the named surface and leaving another open
one frame further out. One of those defeats passed because the assertion
matched a substring that **vitest's own diff output** supplies, so it
held whether or not production code produced it. Text-span pinning
cannot close that class by extension, so it is out of this PR rather
than hardened a fifth time. The replacement — a fetch-stubbed harness
that injects a family, drives the real chain, and asserts a report is
emitted — belongs in #349, where silencing at any frame fails it.

**`FROZEN` extraction is lazy.** It ran at module scope, so one renamed
surface threw at import and took out every other pin, every behavioural
anchor and the whole frozen-data block with it — 0 of 37 tests running.
The obvious repair a future reader reaches for is deleting the offending
entry, which would leave everything else silently unguarded. Failures
are now contained to their own case.

**Live provider canaries no longer run in the offline suite.**
`assertNoUnclassifiedFamilies` moved to a plain `text-drift.ts` with no
`describe` blocks, so importing it no longer executes live specs. `pnpm
test` is 4767 passed / **0 skipped**, and running the full offline suite
with a deliberately invalid key produces no request to `api.openai.com`.
This leak was **pre-existing** — one offline test already imported the
live module before this branch existed — and is only partly this PR's to
fix; what is fixed is that nothing in the offline suite reaches the
network at this head.

Also corrected: the frozen-surface inventory was wrong in both
directions, an anchor claim covering 6 of 12 entries was narrowed to
what it actually covers, and a test titled "still propagates" now
actually calls the function it claims to exercise.

**Deferred, deliberately:** other offline regressions still living in
`models.drift.ts` run only under `test:drift`. Moving them is a
~440-line change and out of scope here — that kind of accretion is what
required this PR to be split in the first place.


---

## Round-5

- **Anti-silence anchor removed** and moved to #349, per the reasoning
above. `logic-pin.test.ts` goes 37 → 29 tests. Three pin hashes deleted,
**zero added** — nothing was re-pinned on the way out. `text-drift.ts`'s
docstring now records that wrapper as *knowingly unguarded pending #349*
rather than silently dropping a protection claim.
- **The mirror-equivalence guard now catches all three divergences its
own docstring names.** Two of the three previously passed when broken —
no fixture contained a non-model token, and every output happened to
already be sorted, so dropping either protection left the suite green.
Two fixtures fix it, each with its own demonstrated failure.
- **The four behavioural classification tests still bind**, re-verified
on this head: dropping either family reds its own test, a substring
match reds on `gpt-live`, a prefix match reds on
`gpt-live-transcribe-mini`.

**Noted for a follow-up, not done here:** a single `import { expect }
from "vitest"` at module scope in `text-drift.ts` is the only thing
preventing `scripts/drift-sync.ts` from importing the real module.
Replacing it with a plain `Error` carrying the message the collector
parses would retire the ~440-line hand-maintained mirror entirely. One
line out, 440 deleted.

**Also pre-existing, reported not fixed:** the reference scan excludes
only `__tests__/drift`, so `isFamilyStillReferenced` returns `true` for
never-existing families off fixture literals alone. It errs toward
over-blocking a removal, so it is fail-safe rather than dangerous.

Both PR-open steps dedup by matching HTML comments they themselves wrote into a
PR BODY — machine state parked inside prose a human owns and edits. On
2026-08-03 a human rewrote PR #343's body wholesale, deleting its markers; ~14h
later the scheduled run could no longer find the changeset key on any open PR
and opened duplicate PR #350 for the same changeset, which is exactly the spam
those guards exist to prevent.

Validating-and-failing would have surfaced it a run sooner and STILL left a
human to repair prose a human will keep editing, so each step now RE-ASSERTS its
own markers before dedup:

- ONE `MARKERS` array is the single source for the marker set (`MARKER` seeds
  it, `NOTE_MARKERS[i]` appends one per committed note). The body writer emits
  every entry, the repair re-asserts every entry, and the per-note guard matches
  `NOTE_MARKERS[$i]` — so a marker added there is wired into all three at once.
  A repair that covered `drift-changeset:` but not `drift-proposal-note:` would
  leave the second marker's dedup path just as defeatable.
- Candidates come from a plain `--state open` listing, never `--search`: that
  index is body-keyed AND lags an edit by minutes, so it cannot see the body
  about to be repaired.
- Identity is body-INDEPENDENT, since a body-keyed lookup cannot find a PR whose
  body lost the key. Anchor (a): the head branch now ENDS in the changeset key,
  and a PR's head branch cannot be renamed — this covers the mixed run whose
  diff carries no note file. Anchor (b): the PR's changed files include one of
  this run's note paths — this covers a re-fire whose outcome set changed and so
  hashes to a DIFFERENT key, where (a) cannot match and the per-note marker is
  the only thing preventing a duplicate.
- The repair is anchor-SCOPED. A PR matched only by a shared note gets that
  note's marker and nothing else; stamping this run's changeset key into it would
  assert a changeset it does not carry (in a mixed run, the registry-edit half)
  and suppress that half forever.
- Append-only after the existing body, and a body with every marker intact is
  left byte-identical: no edit, no API write, no marker duplicated by a re-run.
- Repaired PRs are folded into the dedup candidate set, since `--search` cannot
  see a body edited seconds ago — repairing without that would still open the
  duplicate the repair just prevented.

Fails closed throughout: an unreadable PR list or a failed edit exits non-zero
rather than encoding "could not look" as "nothing to repair". No dedup guard is
weakened or bypassed.